Abstract: In this paper, we present TEST, an approach that automatically generates unit tests for object-oriented classes based on mutation analysis. By using mutations rather than structural properties as coverage criterion, we not only get guidance in where to test, but also what to test for. This allows us to generate effective test oracles, a feature raising automation to a level not offered by traditional tools.To assess the quality of test suites, mutation analysis seeds artificial defects (mutations) into programs; a non-detected mutation indicates a weakness in the test suite. We present an automated approach to generate unit tests that detect these mutations for object-oriented classes. This has two advantages: First, the resulting test suite is optimized toward finding defects modelled by mutation operators rather than covering code. Second, the state change caused by mutations induces oracles that precisely detect the mutants. Evaluated on 10 open source libraries, our _TEST prototype generates test suites that find significantly more seeded defects than the original manually written test suites. Improving test cases after mutation analysis usually means that the tester has to go back to the drawing-board and design new test cases, taking the feedback gained from the mutation analysis into account. This process requires a deep understanding of the source code and is a nontrivial task. Automated test generation can help in covering code (and thus hopefully detecting mutants), but even then, the tester still needs to assess the results of the generated executions—and has to write hundreds or even thousands of oracles. The test cases of an existing test suite are executed on a program version (mutant) containing one such fault at a time in order to see if any of the test cases can detect that there is a fault. A mutant that is detected as such is considered dead and of no further use. A live mutant, however, shows a case where the test suite potentially fails to detect an error and therefore needs improvement. There are two main problems of mutation analysis: First, the sheer number of mutants and the effort of checking all tests against all mutants can cause significant computational costs. Second, equivalent mutants do not observably change the program behaviour or are even semantically identical, and so there is no way to possibly detect them by testing.

Keywords: Mutation analysis, Testing, Object-Oriented Classes, Test Generation.